Pattern angle spatial selection structured illumination imaging

ABSTRACT

The disclosure provides for structured illumination microscopy (SIM) imaging systems. In one set of implementations, a SIM imaging system may be implemented as a multi-arm SIM imaging system, whereby each arm of the system includes a light emitter and a beam splitter (e.g., a transmissive diffraction grating) having a specific, fixed orientation with respect to the system&#39;s optical axis. In a second set of implementations, a SIM imaging system may be implemented as a multiple beam splitter slide SIM imaging system, where one linear motion stage is mounted with multiple beam splitters having a corresponding, fixed orientation with respect to the system&#39;s optical axis. In a third set of implementations, a SIM imaging system may be implemented as a pattern angle spatial selection SIM imaging system, whereby a fixed two-dimensional diffraction grating is used in combination with a spatial filter wheel to project one-dimensional fringe patterns on a sample.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a 35 U.S.C. 371 National Stage of InternationalPatent Application No. PCT/US2019/013504, filed Jan. 14, 2019, whichitself claims the benefit of and priority to U.S. Provisional PatentApplication 62/618,059, filed Jan. 16, 2018 and Dutch Patent ApplicationNo. N2020620, filed Mar. 20, 2018, the content of each of which isincorporated by reference herein in its entirety and for all purposes.

BACKGROUND

Structured illumination microscopy (SIM) describes a technique by whichspatially structured (i.e., patterned) light may be used to image asample to increase the lateral resolution of the microscope by a factorof two or more. In some instances, during imaging of the sample, threeimages of fringe patterns of the sample are acquired at various patternphases (e.g., 0°, 120°, and 240°), so that each location on the sampleis exposed to a range of illumination intensities, with the procedurerepeated by rotating the pattern orientation about the optical axis to 3separate angles (e.g. 0°, 60° and 120°). The captured images (e.g., nineimages) may be assembled into a single image having an extended spatialfrequency bandwidth, which may be retransformed into real space togenerate an image having a higher resolution than one captured by aconventional microscope.

In some implementations of current SIM systems, a linearly polarizedlight beam is directed through an optical beam splitter that splits thebeam into two or more separate orders that may be combined and projectedon the imaged sample as an interference fringe pattern with a sinusoidalintensity variation. Diffraction gratings are examples of beam splittersthat can generate beams with a high degree of coherence and stablepropagation angles. When two such beams are combined, the interferencebetween them can create a uniform, regularly-repeating fringe patternwhere the spacing is determined by factors including the angle betweenthe interfering beams. If more than two beams are combined, theresulting pattern typically contains a mixture of fringe spacings, withthe result that the difference between the maximum and minimumintensities (also known as the “modulation depth”) is reduced, making itless suitable for SIM purposes.

In some implementations of current SIM systems, the orientation of theprojected pattern is controlled by rotating the beam splitting elementabout the optic axis, and the phase of the pattern is adjusted by movingthe element laterally across the axis. In such systems, a diffractiongrating is typically mounted on a translation stage, which in turn ismounted on a rotation stage. Additionally, such systems often utilize alinear polarizer to polarize the light emitted by the light sourcebefore it is received at the grating.

SUMMARY

Implementations disclosed herein are directed to structured illuminationsystems and methods.

In a first set of implementations, a SIM imaging system may beimplemented as a multi-arm SIM imaging system, where each arm of thesystem includes a light emitter and a beam splitter (e.g., atransmissive diffraction grating) having a specific, fixed orientationwith respect to the system's optical axis.

In one implementation of a multi-arm SIM imaging system, the systemincludes: a first optical arm, including: a first light emitter to emitlight; and a first beam splitter to split light emitted by the firstlight emitter to project a first plurality of fringes on a plane of asample; and a second optical arm, including: a second light emitter toemit light; and a second beam splitter to split light emitted by thesecond light emitter to project a second plurality of fringes on theplane of the sample. In this implementation, the system may also includean optical element to combine an optical path of the first arm and thesecond arm. Additionally, the system may include an image sensor tocollect light emitted by the sample. In some implementations, the samplemay include a plurality of features regularly patterned in a rectangulararray or hexagonal array.

In some implementations, the first beam splitter includes a firsttransmissive diffraction grating and the second beam splitter includes asecond transmissive diffraction grating. In some implementations, thefirst beam splitter includes a first reflective diffraction grating andthe second beam splitter includes a second reflective diffractiongrating. In some implementations, the first and second beam splitterseach include a beam splitter cube or plate.

In some implementations, the first and second light emitters emitunpolarized light, and the first and second transmissive diffractiongratings are to diffract unpolarized light emitted by a respective oneof the first and second light emitters.

In some implementations, the optical element to combine an optical pathof the first plurality of fringes and the second plurality of fringesincludes a mirror with holes, with the mirror arranged to reflect lightdiffracted by the first diffraction grating and with the holes arrangedto pass through at least first orders of light diffracted by the seconddiffraction grating. In some implementations, the optical element tocombine an optical path of the first arm and the second arm includes apolarizing beam splitter, where the first diffraction grating diffractsvertically polarized light and where the second diffraction gratingdiffracts horizontally polarized light.

In some implementations, the multi-arm SIM imaging system includes oneor more optical elements to phase shift the first plurality of fringesand the second plurality of fringes.

In some implementations, the one or more optical elements to phase shiftthe first plurality of fringes and the second plurality of fringesinclude a first rotating optical window to phase shift the firstplurality of fringes and a second rotating optical window to phase shiftthe second plurality of optical fringes. In some implementations, theone or more optical elements to phase shift the first plurality offringes and the second plurality of fringes include a first linearmotion stage to translate the first diffraction grating and a secondlinear motion stage to translate the second diffraction grating. In someimplementations, the one or more optical elements to phase shift thefirst plurality of fringes and the second plurality of fringes include asingle rotating optical window, where the single rotating optical windowis positioned after the mirror with holes in an optical path to thesample.

In some implementations, an axis of rotation of the single rotatingoptical window is offset by about 45 degrees from an optical axis ofeach of the gratings.

In some implementations, the first plurality of fringes are angularlyoffset from the second plurality of fringes on the sample plane by about90 degrees.

In some implementations, the system also includes: an objective lens toproject each of the first plurality of fringes and the second pluralityof fringes on the sample.

In some implementations, the system also includes: one or more opticalbeam blockers for blocking zero orders of light emitted by each of thefirst and second diffraction gratings. In particular implementations,the one or more optical beam blocks include a Bragg grating.

In one implementation of a multi-arm SIM imaging system, a methodincludes: turning on a first optical arm of a structured illuminationsystem, the first optical arm comprising a first light emitter to emitlight and a first diffraction grating to diffract light emitted by thefirst light emitter to project a first plurality of fringes oriented ina specific direction on a plane of a sample; capturing a first pluralityof phase images of the sample, where during capture of the firstplurality of images, the positions of the first plurality of fringes areshifted on the plane of the sample; turning on a second optical arm ofthe structured illumination system, the second optical arm comprising asecond light emitter to emit light and a second diffraction grating todiffract light emitted by the second light emitter to project a secondplurality of fringes on the plane of the sample, where the secondplurality of fringes are angularly offset from the first plurality offringes on the plane of the sample; and capturing a second plurality ofphase images of the sample illuminated with the second plurality offringes, where during capture of the second plurality of fringes, thepositions of the second plurality of fringes are shifted on the plane ofthe sample. In implementations of this method, the first diffractiongrating and the second diffraction grating are transmissive diffractiongratings, where the structured illumination system includes a mirrorwith holes to reflect light diffracted by the first diffraction gratingand to pass through at least first orders of light diffracted by thesecond diffraction grating.

In implementations, the method further includes: using at least thefirst plurality of captured phase images and the second plurality ofcaptured phased images to computationally reconstruct one or more imageshaving higher resolution than each of the first and second pluralitiesof captured phased images. In implementations, the first plurality offringes are angularly offset from the second plurality of fringes on thesample plane by about 90 degrees.

In implementations, the first plurality of fringes and the secondplurality of fringes are phase shifted by rotating a single opticalwindow positioned in an optical path between the sample and each of thefirst and second gratings, where an axis of rotation of the singlerotating optical window is offset from an optical axis of each of thegratings.

In implementations of the method, the first optical arm is turned offand the second optical arm of the structured illumination system isturned on after capturing the first plurality of phase images.

In implementations of the method, the first diffraction grating and thesecond diffraction grating are mechanically fixed during image capture.

In a second set of implementations, a SIM imaging system may beimplemented as a multiple beam splitter slide SIM imaging system, whereone linear motion stage is mounted with multiple beam splitters having acorresponding, fixed orientation with respect to the system's opticalaxis.

In one implementation of a multiple beam splitter slide SIM imagingsystem, the system includes: a light emitter to emit light; a linearmotion stage mounted with a first beam splitter and a second beamsplitter, where the first beam splitter is to split light emitted by thelight emitter to project a first plurality of fringes on a plane of asample, and where the second beam splitter is to split light emitted bythe light emitter to project a second plurality of fringes on the planeof the sample; and an image sensor to collect light emitted by thesample. In implementations, the linear motion stage is a one-dimensionallinear motion stage, where the linear motion stage is to translate alongthe one dimension to optically couple each of the first beam splitterand the second beam splitter to the light emitter, where the first beamsplitter is adjacent to the second beam splitter along the onedimension. In implementations, the first plurality of fringes areangularly offset from the second plurality of fringes on the sampleplane by about 90 degrees.

In implementations, the first beam splitter includes a firsttransmissive diffraction grating and the second beam splitter includes asecond transmissive diffraction grating. The first diffraction gratingand the second diffraction grating may be angularly offset from the onedimension (i.e., rotated around the propagation direction of light). Inparticular implementations, the first diffraction grating and the seconddiffraction grating are angularly offset from the one dimension by about±45 degrees.

In some implementations, the first diffraction grating and the seconddiffraction grating may be integrated into a single optical elementmounted on the linear motion stage. In implementations where thediffraction gratings are integrated into a single optical element, thesingle optical element may include a first side patterned with the firstdiffraction grating and a second side, adjacent the first side,patterned with the second diffraction grating.

In some implementations, the system may further include: one or moreoptical beam blockers for blocking zero orders of light emitted by eachof the first and second diffraction gratings.

In some implementations, the system may further include: a projectionlens in an optical path between the linear motion stage and theobjective lens. The projection lens may be to project a Fouriertransform of each of the first diffraction grating and the seconddiffraction into an entrance pupil of the objective.

In some implementations, the system may further include an alignmentpattern formed on a component mounted on the linear motion stage, wherethe alignment pattern splits light emitted by the light emitter toproject a pattern on the plane of the sample for imaging alignment. Thealignment pattern may be formed on a substrate including at least one ofthe first diffraction grating and the second diffraction grating. Theprojected pattern may include lines having a lower frequency than theprojected first plurality of fringes and second plurality of fringes.

In some implementations, the system may further include: an opticalphase modulator to phase shift the first plurality of fringes and secondplurality of fringes that are projected on the plane of the sample. Insuch implementations, the optical phase modulator may be a separatecomponent from the linear motion stage.

In one implementation of a multiple beam splitter slide SIM imagingsystem, a method includes: turning on a light emitter of a structuredillumination imaging system, the structured illumination imaging systemincluding a one-dimensional linear motion stage mounted with a firstdiffraction grating and a second diffraction grating, where the linearmotion stage is to translate along one dimension; translating the linearmotion stage along the one dimension to phase shift a first plurality offringes projected by the first diffraction grating on a sample;translating the linear motion stage to optically couple the seconddiffraction grating to the light emitter; and after optically couplingthe second diffraction grating to the light emitter, translating thelinear motion stage along the one dimension to phase shift a secondplurality of fringes projected by the second diffraction grating on thesample. The first diffraction grating and the second diffraction gratingmay be transmissive diffraction gratings and may be angularly offsetfrom the one dimension of translation. For example, the firstdiffraction grating and the second diffraction grating may be angularlyoffset from the one dimension by about ±45 degrees.

In implementations, the method may further include: translating thelinear motion stage along the one dimension a plurality of times tophase shift, a plurality of times, the first plurality of fringesprojected by the first diffraction grating on the sample; and afteroptically coupling the second diffraction grating to the light emitter,translating the linear motion stage along the one dimension a pluralityof times to phase shift, a plurality of times, the second plurality offringes projected by the second diffraction grating on the sample.

In implementations, the method may further include: capturing an imageof the sample after each time that the linear motion stage is translatedto phase shift the first plurality of fringes; and capturing an image ofthe sample after each time that the linear motion stage is translated tophase shift the second plurality of fringes. The captured images may beused to computationally reconstruct an image having a higher resolutionthan each of the captured images.

In implementations of the method, the linear motion stage is translatedabout the same distance along the one dimension each time the firstplurality of fringes or the second plurality of fringes are phasedshifted on the sample.

In particular implementations, the linear motion is stage is translatedbetween about 10 mm and 15 mm when the second diffraction grating isoptically coupled to the light emitter.

In a third set of implementations, a SIM imaging system may beimplemented as a pattern angle spatial selection SIM imaging system,whereby a fixed two-dimensional diffraction grating is used incombination with a spatial filter wheel to project one-dimensionalfringe patterns on a sample.

In one implementation of a pattern angle spatial selection SIM imagingsystem, the system includes: a light emitter to emit light; atwo-dimensional diffraction grating to diffract light emitted by thelight emitter to project a first plurality of fringes oriented in afirst direction on a sample plane and to project a second plurality offringes oriented in a second direction, perpendicular to the firstdirection, on the sample plane; and a spatial filter wheel to passthrough diffracted light received from the two-dimensional diffractiongrating in a respective one of the first or second directions and blocklight in a respective one of the first or second directions, the spatialfilter wheel comprising a first plurality of apertures and a secondplurality of apertures orthogonal to the first plurality of apertures.The first plurality of apertures may be to pass through light diffractedby the two-dimensional diffraction in the first direction and the secondplurality of apertures may be to pass through light diffracted by thetwo-dimensional diffraction in the second direction.

In some implementations, the system further includes: a beam blockingelement to block 0th order light transmitted by the two-dimensionaldiffraction grating. In particular implementations, the beam blockingelement includes a diffractive optical element patterned to reflectlight normal to the element and pass through light at other angles.

In some implementations, the spatial filter wheel is to reflectdiffraction orders of light received from the two-dimensionaldiffraction grating that are not passed through.

In some implementations, the two-dimensional diffraction grating is atransmissive diffraction grating. The transmissive diffraction gratingmay be disposed over or formed on a face of a solid optic that receiveslight from the light emitter. Dispersion angles of the transmissivediffraction grating may be arranged such that 0th order light is blockedon a far side of the solid optic. In some implementations, the solidoptic includes angled faces to diffract and output first orders of lightdiffracted by the two-dimensional transmissive diffraction grating. Inparticular implementations, the angled faces include a focusing lens. Insome implementations, a projection lens receives light output by thesolid optic.

In some implementations, the two-dimensional diffraction grating is atwo-dimensional reflective diffraction grating. The two-dimensionalreflective diffraction grating may be disposed over or formed on a faceof the solid optic opposite an aperture of the solid optic that receiveslight from the light emitter. The solid optic may include reflectiveinternal faces to reflect and output first orders of light diffracted bythe two-dimensional reflective diffraction grating through outlet facesof the solid optic. In particular implementations, the outlet facesinclude a diffractive focusing lens. In some implementations, aprojection lens is to receive light output by the solid optic.

In some implementations, the system further includes: one or moreoptical elements to phase shift the first plurality of fringes and thesecond plurality of fringes. In particular implementations, the one ormore optical elements to phase shift the first plurality of fringes andthe second plurality of fringes comprise a parallel plate optic tiltedin two perpendicular directions.

In one implementation of a pattern angle spatial selection SIM imagingsystem, a method includes: turning on a light emitter of a structuredillumination imaging system, the structured illumination imaging systemincluding a two-dimensional diffraction grating; receiving light emittedby the light emitter at the two-dimensional diffraction grating tooutput first diffracted light oriented in a first direction and seconddiffracted light oriented in a second direction perpendicular to thefirst direction; passing the first diffracted light through a firstplurality of apertures of a spatial filter wheel and blocking the seconddiffracted light at the spatial filter wheel; projecting the firstdiffracted light the passed through the first plurality of holes as afirst plurality of fringes on a sample plane; and capturing a firstplurality of phase images of light emitted by the sample, wherein duringcapture of the first plurality of images, the first plurality of fringesare phase shifted on the sample plane. The first plurality of fringesmay be phase shifted by moving the sample (e.g., using a motion stage),by moving the projected fringes, or by moving both the sample andprojected fringes.

In implementations, the method further includes: rotating the spatialfilter wheel such that it passes the second diffracted light through asecond plurality of apertures of the spatial filter wheel and blocks thefirst diffracted light at the spatial filter wheel; projecting thesecond diffracted light that passes through the second plurality ofholes as a second plurality of fringes, orthogonal to the firstplurality of fringes, on the sample plane; and capturing a secondplurality of phase images of light emitted by the sample, where duringcapture of the second plurality of images, the second plurality offringes are phase shifted on the sample plane.

In particular implementations of the method, the two-dimensionaldiffraction grating is a two-dimensional transmissive diffractiongrating formed on or disposed over a face of a solid optic, and themethod further includes: blocking 0th order light output by thetransmissive diffraction grating at a side of the solid optic oppositethe transmissive diffraction grating; and diffracting and outputting,from angled faces of the solid optic, first orders of light diffractedby the two-dimensional transmissive diffraction grating.

In particular implementations of the method, the two-dimensionaldiffraction grating is a two-dimensional reflective diffraction gratingformed on or disposed over a face of a solid optic opposite an apertureof the solid optic that receives light from the light emitter, and themethod further includes: reflecting, at faces of the solid optic, firstorders of light diffracted by the two-dimensional reflective diffractiongrating.

Other features and aspects of the disclosed technology will becomeapparent from the following detailed description, taken in conjunctionwith the accompanying drawings, which illustrate, by way of example, thefeatures in accordance with some implementations described herein of thedisclosed technology. The summary is not intended to limit the scope ofany inventions described herein, which are defined by the claims andequivalents.

It should be appreciated that all combinations of the foregoing concepts(provided such concepts are not mutually inconsistent) are contemplatedas being part of the inventive subject matter disclosed herein. Inparticular, all combinations of claimed subject matter appearing at theend of this disclosure are contemplated as being part of the inventivesubject matter disclosed herein.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure, in accordance with one or more implementations,is described in detail with reference to the following figures. Thefigures are provided for purposes of illustration only and merely depictexample implementations. Furthermore, it should be noted that forclarity and ease of illustration, the elements in the figures have notnecessarily been drawn to scale.

Some of the figures included herein illustrate various implementationsof the disclosed technology from different viewing angles. Although theaccompanying descriptive text may refer to such views as “top,” “bottom”or “side” views, such references are merely descriptive and do not implyor require that the disclosed technology be implemented or used in aparticular spatial orientation unless explicitly stated otherwise.

FIG. 1 illustrates a structured illumination imaging system thatilluminates a sample with spatially structured light, in accordance withsome implementations described herein.

FIG. 2 is an optical diagram illustrating one example opticalconfiguration of a two-arm structured illumination microscopy (SIM)imaging system, in accordance with some implementations describedherein.

FIG. 3 is an optical diagram illustrating another example opticalconfiguration of a two-arm SIM imaging system, in accordance with someimplementations described herein.

FIG. 4 is an optical diagram illustrating another example opticalconfiguration of a two-arm SIM imaging system, in accordance with someimplementations described herein.

FIG. 5 is an operational flow diagram illustrating an example methodthat may be performed by a multi-arm SIM imaging system during oneimaging cycle to use structured light to create a high resolution image,in accordance with some implementations described herein.

FIG. 6 illustrates simplified illumination fringe patterns that may beprojected onto the plane of a sample by a vertical grating andhorizontal grating of a two-arm SIM imaging system during image capture,in accordance with some implementations described herein.

FIG. 7 illustrates an example experimental design of a two-arm SIMimaging system that uses a polarizing beam splitter to illuminate avertical grating with vertically-polarized light and a horizontalgrating with horizontally-polarized light, in accordance with someimplementations described herein.

FIG. 8A illustrates an afocal mirror image and fluorescent slidecaptured using the example SIM imaging system of FIG. 7, using a20×/0.75 NA microscope.

FIG. 8B illustrates fringe modulation measurements acquired using thesystem of FIG. 7 with a beaded flowcell. The graph illustrates typicalfeature image intensity changes during a phase adjustment cycle in thisexample, as the angle of parallel plate W2 of FIG. 7 is changed.

FIG. 9 illustrates another example optical configuration of a two-armSIM imaging system in accordance with some implementations describedherein.

FIG. 10A is a schematic diagram illustrating an example opticalconfiguration of a dual optical grating slide SIM imaging system in afirst diffraction grating position, in accordance with someimplementations described herein.

FIG. 10B is a schematic diagram illustrating an example opticalconfiguration of the dual optical grating slide SIM imaging system ofFIG. 10A in a second diffraction grating position, in accordance withsome implementations described herein.

FIG. 11 is an operational flow diagram illustrating an example methodthat may be performed by a multiple optical grating slide SIM imagingsystem during one imaging cycle to use structured light to create a highresolution image, in accordance with some implementations describedherein.

FIG. 12 illustrates simplified illumination fringe patterns that may beprojected onto the plane of a sample by a first diffraction grating anda second diffraction grating of a dual optical grating slide SIM imagingsystem during image capture, in accordance with some implementationsdescribed herein.

FIG. 13 is a diagram illustrating an example dual optical grating slideSIM imaging configuration in accordance with some implementationsdescribed herein.

FIG. 14 is a schematic diagram illustrating an example opticalconfiguration of a pattern angle spatial selection SIM imaging system,in accordance with some implementations described herein.

FIG. 15 is a schematic diagram illustrating another example opticalconfiguration of a pattern angle spatial selection SIM imaging system,in accordance with some implementations described herein.

FIG. 16 is a schematic diagram illustrating another example opticalconfiguration of a pattern angle spatial selection SIM imaging system,in accordance with some implementations described herein.

FIG. 17 shows one example of an alignment pattern that may be used insome implementations of a multiple optical grating slide SIM imagingsystem.

FIG. 18 illustrates a sample that may be formed over an image sensorassembly of a SIM imaging system, in accordance with someimplementations described herein.

FIG. 19 illustrates some components of an example dual optical gratingslide SIM imaging system in accordance with some implementationsdescribed herein.

The figures are not exhaustive and do not limit the present disclosureto the precise form disclosed.

DETAILED DESCRIPTION

As used herein to refer to light diffracted by a diffraction grating,the term “order” or “order number” is intended to mean the number ofinteger wavelengths that represents the path length difference of lightfrom adjacent slits or structures of the diffraction grating forconstructive interference. The interaction of an incident light beam ona repeating series of grating structures or other beam splittingstructures can redirect or diffract portions of the light beam intopredictable angular directions from the original beam. The term “zerothorder” or “zeroth order maximum” is intended to refer to the centralbright fringe emitted by a diffraction grating in which there is nodiffraction. The term “first-order” is intended to refer to the twobright fringes diffracted to either side of the zeroth order fringe,where the path length difference is ±1 wavelengths. Higher orders arediffracted into larger angles from the original beam. The properties ofthe grating can be manipulated to control how much of the beam intensityis directed into various orders. For example, a phase grating can befabricated to maximize the transmission of the ±1 orders and minimizethe transmission of the zeroth order beam.

As used herein to refer to a sample, the term “feature” is intended tomean a point or area in a pattern that can be distinguished from otherpoints or areas according to relative location. An individual featurecan include one or more molecules of a particular type. For example, afeature can include a single target nucleic acid molecule having aparticular sequence or a feature can include several nucleic acidmolecules having the same sequence (and/or complementary sequence,thereof).

As used herein, the term “xy plane” is intended to mean a 2-dimensionalarea defined by straight line axes x and y in a Cartesian coordinatesystem. When used in reference to a detector and an object observed bythe detector, the area can be further specified as being orthogonal tothe beam axis, or the direction of observation between the detector andobject being detected.

As used herein, the term “z coordinate” is intended to mean informationthat specifies the location of a point, line or area along an axis thatis orthogonal to an xy plane. In particular implementations, the z axisis orthogonal to an area of an object that is observed by a detector.For example, the direction of focus for an optical system may bespecified along the z axis.

As used herein, the term “optically coupled” is intended to refer to oneelement being adapted to impart light to another element directly orindirectly.

As noted above, pre-existing implementations of SIM systems mount adiffraction grating on a translation stage, which in turn is mounted ona rotation stage. Additionally, such systems often utilize a linearpolarizer for polarizing the light source before it is received at thegrating. This pre-existing design suffers from a number of drawbacks foruse in a high-throughput microscopy system. First, because a rotationstage must rotate the grating several times during acquisition of animage set (e.g., three times), this slows down the instrument's speedand affects its stability. Typically, the fastest grating stages canrotate is on the order of tens of milliseconds (ms), which imposes amechanical throughput limit on imaging speed. Second, the pre-existingdesign has poor repeatability because mechanical tolerances of therotation stage limit the repeatability of the structured illuminationpatterns from one image acquisition set to the next. This also imposes ahigher cost on the SIM system as it requires a very precise rotationstage.

Third, the pre-existing SIM design is not the most reliable for use in ahigh-throughput microscopy system because of the number of actuationsthat are made to rotate the grating. For example, if one SIM image setis acquired every second, the rotation stage may require millions totens of millions actuations per year. Fourth, the pre-existing SIMdesign has low optical efficiency because the linear polarizer blocks atleast 50% of the light received at the grating.

To this end, implementations of the technology disclosed herein aredirected to improved SIM systems and methods.

In accordance with a first set of implementations of the technologydisclosed herein, a SIM imaging system may be implemented as a multi-armSIM imaging system, whereby each arm of the system includes a lightemitter and a beam splitter (e.g., a transmissive diffraction grating)having a specific, fixed orientation with respect to the optical axis ofthe system. In accordance with these implementations, the beam splittersin the SIM imaging system are rotatably fixed (i.e., do not requiremechanical rotation), which may provide improved system speed,reliability, and repeatability. For systems where the objects beingimaged are oriented primarily along 2 perpendicular axes (i.e. verticaland horizontal), it is possible to achieve enhanced spatial resolutionusing 2 pattern angles, instead of the 3 angles typically used forrandomly-oriented objects. In particular implementations, the system maybe implemented as a two-arm SIM imaging system including a fixedvertical grating and a fixed horizontal grating to project respectivefringe patterns on an imaged sample. Other pairs of orthogonal gratingand pattern angles can be used, provided they are aligned with theorientation of sample objects. Additionally, the system may include amirror with holes to combine the two arms into the optical path in alossless manner.

In accordance with a second set of implementations of the technologydisclosed herein, a SIM imaging system may be implemented as a multiplebeam splitter slide SIM imaging system, where one linear motion stage ismounted with a plurality of beam splitters (e.g., diffraction gratings)having a corresponding, fixed orientation with respect to the opticalaxis of the system. In particular implementations, the SIM imagingsystem may be implemented as a dual optical grating slide SIM imagingsystem whereby all phase shifts or rotations of the grating patternprojected on imaged sample may be made by linearly translating a motionstage along a single axis of motion, to select one of two gratings or toeffect a phase shift of the pattern generated by a selected grating. Insuch implementations, only a single optical arm having a single emitterand single linear motion stage is needed to illuminate a sample, whichmay provide system advantages such as reducing the number of movingsystem parts to improve speed, complexity and cost. Additionally, insuch implementations, the absence of a polarizer may provide theadvantage of high optical efficiency.

In accordance with a third set of implementations of the technologydisclosed herein, a SIM imaging system may be implemented as a patternangle spatial selection SIM imaging system, whereby a fixedtwo-dimensional diffraction grating is used in combination with aspatial filter wheel to project one-dimensional diffraction patterns ona sample. In such implementations, the primary optical components of theimaging system may remain stationary, which may improve the stability ofthe optical system (and of the illumination pattern) and minimize theweight, vibration output, and cost of the moving elements of the system.

Before describing various implementations of the systems and methodsdisclosed herein, it is useful to describe an example environment withwhich the technology disclosed herein can be implemented. One suchexample environment is that of a structured illumination imaging system100, illustrated in FIG. 1, that illuminates a sample with spatiallystructured light. For example, system 100 may be a structuredillumination fluorescence microscopy system that utilizes spatiallystructured excitation light to image a biological sample.

In the example of FIG. 1, a light emitter 150 is configured to output alight beam that is collimated by collimation lens 151. The collimatedlight is structured (patterned) by light structuring optical assembly155 and directed by dichroic mirror 160 through objective lens 142 ontoa sample of a sample container 110, which is positioned on a motionstage 170. In the case of a fluorescent sample, the sample fluoresces inresponse to the structured excitation light, and the resultant light iscollected by objective lens 142 and directed to an image sensor ofcamera system 140 to detect fluorescence.

Light structuring optical assembly 155 in various implementations,further described below, includes one or more optical diffractiongratings or other beam splitting elements (e.g., a beam splitter cube orplate) to generate a pattern of light (e.g., fringes, typicallysinusoidal) that is projected onto samples of a sample container 110.The diffraction gratings may be one-dimensional or two-dimensionaltransmissive or reflective gratings. The diffraction gratings may besinusoidal amplitude gratings or sinusoidal phase gratings.

As further described below with reference to particular implementations,in system 100 the diffraction gratings do not require a rotation stagelike the typical structured illumination microscopy system ofpreexisting systems discussed above. In some implementations, thediffraction gratings may be fixed during operation of the imaging system(i.e., not require rotational or linear motion). For example, in aparticular implementation, further described below, the diffractiongratings may include two fixed one-dimensional transmissive diffractiongratings oriented perpendicular to each other (e.g., a horizontaldiffraction grating and vertical diffraction grating).

As illustrated in the example of FIG. 1, light structuring opticalassembly 155 outputs the first orders of the diffracted light beams(e.g., m=±1 orders) while blocking or minimizing all other orders,including the zeroth orders. However, in alternative implementations,additional orders of light may be projected onto the sample.

During each imaging cycle, imaging system 100 utilizes light structuringoptical assembly 155 to acquire a plurality of images at various phases,with the fringe pattern displaced laterally in the modulation direction(e.g., in the x-y plane and perpendicular to the fringes), with thisprocedure repeated one or more times by rotating the pattern orientationabout the optical axis (i.e., with respect to the x-y plane of thesample). The captured images may then be computationally reconstructedto generate a higher resolution image (e.g., an image having about twicethe lateral spatial resolution of individual images).

In system 100, light emitter 150 may be an incoherent light emitter(e.g., emit light beams output by one or more excitation diodes), or acoherent light emitter such as emitter of light output by one or morelasers or laser diodes. As illustrated in the example of system 100,light emitter 150 includes an optical fiber 152 for guiding an opticalbeam to be output. However, other configurations of a light emitter 150may be used. In implementations utilizing structured illumination in amulti-channel imaging system (e.g., a multi-channel fluorescencemicroscope utilizing multiple wavelengths of light), optical fiber 152may optically couple to a plurality of different light sources (notshown), each light source emitting light of a different wavelength.Although system 100 is illustrated as having a single light emitter 150,in some implementations multiple light emitters 150 may be included. Forexample, multiple light emitters may be included in the case of astructured illumination imaging system that utilizes multiple arms,further discussed below.

In some implementations, system 100 may include a projection lens 156that may include a lens element to articulate along the z-axis to adjustthe structured beam shape and path. For example, a component of theprojection lens may be articulated to account for a range of samplethicknesses (e.g., different cover glass thickness) of the sample incontainer 110.

In the example of system 100, fluid delivery module or device 190 maydirect the flow of reagents (e.g., fluorescently labeled nucleotides,buffers, enzymes, cleavage reagents, etc.) to (and through) samplecontainer 110 and waste valve 120. Sample container 110 can include oneor more substrates upon which the samples are provided. For example, inthe case of a system to analyze a large number of different nucleic acidsequences, sample container 110 can include one or more substrates onwhich nucleic acids to be sequenced are bound, attached or associated.The substrate can include any inert substrate or matrix to which nucleicacids can be attached, such as for example glass surfaces, plasticsurfaces, latex, dextran, polystyrene surfaces, polypropylene surfaces,polyacrylamide gels, gold surfaces, and silicon wafers. In someapplications, the substrate is within a channel or other area at aplurality of locations formed in a matrix or array across the samplecontainer 110. System 100 also may include a temperature stationactuator 130 and heater/cooler 135 that can optionally regulate thetemperature of conditions of the fluids within the sample container 110.

In particular implementations, the sample container 110 may beimplemented as a patterned flow cell including a translucent coverplate, a substrate, and a liquid contained therebetween, and abiological sample may be located at an inside surface of the translucentcover plate or an inside surface of the substrate. The flow cell mayinclude a large number (e.g., thousands, millions, or billions) of wellsor regions that are patterned into a defined array (e.g., a hexagonalarray, rectangular array, etc.) into the substrate. Each region may forma cluster (e.g., a monoclonal cluster) of a biological sample such asDNA, RNA, or another genomic material which may be sequenced, forexample, using sequencing by synthesis. The flow cell may be furtherdivided into a number of spaced apart lanes (e.g., eight lanes), eachlane including a hexagonal array of clusters. Example flow cells thatmay be used in implementations disclosed herein are described in U.S.Pat. No. 8,778,848.

Sample container 110 can be mounted on a sample stage 170 to providemovement and alignment of the sample container 110 relative to theobjective lens 142. The sample stage can have one or more actuators toallow it to move in any of three dimensions. For example, in terms ofthe Cartesian coordinate system, actuators can be provided to allow thestage to move in the X, Y and Z directions relative to the objectivelens. This can allow one or more sample locations on sample container110 to be positioned in optical alignment with objective lens 142.Movement of sample stage 170 relative to objective lens 142 can beachieved by moving the sample stage itself, the objective lens, someother component of the imaging system, or any combination of theforegoing. In some implementations, movement of sample stage 170 may beimplemented during structured illumination imaging to move structuredillumination fringes with respect to the sample to change phases.Further implementations may also include moving the entire imagingsystem over a stationary sample. Alternatively, sample container 110 maybe fixed during imaging.

In some implementations, a focus (z-axis) component 175 may be includedto control positioning of the optical components relative to the samplecontainer 110 in the focus direction (typically referred to as the zaxis, or z direction). Focus component 175 can include one or moreactuators physically coupled to the optical stage or the sample stage,or both, to move sample container 110 on sample stage 170 relative tothe optical components (e.g., the objective lens 142) to provide properfocusing for the imaging operation. For example, the actuator may bephysically coupled to the respective stage such as, for example, bymechanical, magnetic, fluidic or other attachment or contact directly orindirectly to or with the stage. The one or more actuators can beconfigured to move the stage in the z-direction while maintaining thesample stage in the same plane (e.g., maintaining a level or horizontalattitude, perpendicular to the optical axis). The one or more actuatorscan also be configured to tilt the stage. This can be done, for example,so that sample container 110 can be leveled dynamically to account forany slope in its surfaces.

It should be appreciated that although FIG. 1 illustrates the use of anobjective lens 142 for combining and projecting the two beam orders onthe imaged sample as an interference fringe pattern, other suitablemeans may be used to combine the two beams and/or project theinterference pattern on the sample. Any means of redirecting the beamsmay suffice (e.g., using mirrors), provided the path length traversed bythe beams is within a temporal coherence length of the beams.Additionally, in some implementations, the two beam orders mayautomatically overlay for a distance beyond the beam splitter (e.g.,diffraction grating). In such implementations, an interference patternmay appear near the grating, removing the requirement of an additionalprojection system if the diffraction grating is placed sufficientlyclose to the sample. As such, it should be appreciated thatimplementations for SIM described herein may apply to systems that donot rely on objective lens systems to project interference patterns.

The structured light emanating from a test sample at a sample locationbeing imaged can be directed through dichroic mirror 160 to one or moredetectors of camera system 140. In some implementations, a filterswitching assembly 165 with one or more emission filters may beincluded, where the one or more emission filters can be used to passthrough particular emission wavelengths and block (or reflect) otheremission wavelengths. For example, the one or more emission filters maybe used to switch between different channels of the imaging system. In aparticular implementation, the emission filters may be implemented asdichroic mirrors that direct emission light of different wavelengths todifferent image sensors of camera system 140.

Camera system 140 can include one or more image sensors to monitor andtrack the imaging (e.g., sequencing) of sample container 110. Camerasystem 140 can be implemented, for example, as a charge-coupled device(CCD) image sensor camera, but other image sensor technologies such asactive pixel sensors (e.g., complementary metal-oxide-semiconductor(CMOS) image sensors) can be used. In some implementations, structuredillumination imaging system 100 may utilize an image sensor (e.g.,active pixel sensor) in an active plane of the sample. In suchimplementations, the imaged sample may be patterned and/or aligned overthe image sensor.

Output data (e.g., images) from camera system 140 may be communicated toa real-time analysis module (not shown) that may be implemented as asoftware application that, as further described below, may reconstructthe images captured during each imaging cycle to create an image havinga higher spatial resolution. Alternatively, the output data may bestored for reconstruction at a later time.

Although not illustrated, a controller can be provided to control theoperation of structured illumination imaging system 100, includingsynchronizing the various optical components of system 100. Thecontroller can be implemented to control aspects of system operationsuch as, for example, configuration of light structuring opticalassembly 155 (e.g., selection and/or linear translation of diffractiongratings), movement of projection lens 156, focusing, stage movement,and imaging operations. In various implementations, the controller canbe implemented using hardware, algorithms (e.g., machine executableinstructions), or a combination of the foregoing. For example, in someimplementations the controller can include one or more CPUs orprocessors with associated memory. As another example, the controllercan comprise hardware or other circuitry to control the operation, suchas a computer processor and a non-transitory computer readable mediumwith machine-readable instructions stored thereon. For example, thiscircuitry can include one or more of the following: field programmablegate array (FPGA), application specific integrated circuit (ASIC),programmable logic device (PLD), complex programmable logic device(CPLD), a programmable logic array (PLA), programmable array logic (PAL)or other similar processing device or circuitry. As yet another example,the controller can comprise a combination of this circuitry with one ormore processors.

Multi-Arm Structured Illumination Microscopy Imaging System

In accordance with some implementations of the technology disclosedherein, the SIM imaging system may be implemented as a multi-arm SIMimaging system, where each arm of the system includes a light emitterand a grating having a specific, fixed orientation with respect to theoptical axis of the system.

FIG. 2 is an optical diagram illustrating one example opticalconfiguration of a two-arm SIM imaging system 200 in accordance withsome implementations described herein. The first arm of system 200includes a light emitter 210A, an optical collimator 220A to collimatelight output by light emitter 210A, a diffraction grating 230A in afirst orientation with respect to the optical axis, a rotating window240A, and a projection lens 250A. The second arm of system 200 includesa light emitter 210B, an optical collimator 220B to collimate lightoutput by light emitter 210B, a diffraction grating 230B in a secondorientation with respect to the optical axis, a rotating window 240B,and a projection lens 250B. Although diffraction gratings areillustrated in this example, in other implementations, other beamsplitting elements such as a beam splitter cube or plate may be used tosplit light received at each arm of SIM imaging system 200.

Each light emitter 210A-210B may be an incoherent light emitter (e.g.,emit light beams output by one or more light emitting diodes (LEDs)), ora coherent light emitter such as emitter of light output by one or morelasers or laser diodes. In the example of system 200, each light emitter210A-210B is an optical fiber that outputs an optical beam that iscollimated by a respective collimator 220A-220B.

In some implementations, each optical fiber may be optically coupled toa corresponding light source (not shown) such as a laser. Duringimaging, each optical fiber may be switched on or off using a high-speedshutter (not shown) positioned in the optical path between the fiber andthe light source, or by pulsing the fiber's corresponding light sourceat a predetermined frequency during imaging. In some implementations,each optical fiber may be optically coupled to the same light source. Insuch implementations, a beam splitter or other suitable optical elementmay be used to guide light from the light source into each of theoptical fibers. In such examples, each optical fiber may be switched onor off using a high-speed shutter (not shown) positioned in the opticalpath between the fiber and beam splitter.

In example SIM imaging system 200, the first arm includes a fixedvertical grating 230A to project a grating pattern in a firstorientation (e.g., a vertical fringe pattern) onto the sample, and thesecond arm includes a fixed horizontal grating 230B to project a gratingpattern in a second orientation (e.g., a horizontal fringe pattern) ontothe sample 271. Unlike in pre-existing SIM imaging systems, the gratingsof SIM imaging system 200 do not need to be mechanically rotated ortranslated, which may provide improved system speed, reliability, andrepeatability.

As illustrating in the example of FIG. 2, gratings 230A-230B may betransmissive diffraction gratings, including a plurality of diffractingelements (e.g., parallel slits or grooves) formed into a glass substrateor other suitable surface. The gratings may be implemented as phasegratings that provide a periodic variation of the refractive index ofthe grating material. The groove or feature spacing may be chosen todiffract light at suitable angles and tuned to the minimum resolvablefeature size of the imaged samples for operation of SIM imaging system200. In other implementations, the gratings may be reflectivediffraction gratings.

In the example of SIM imaging system 200, the vertical and horizontalpatterns are offset by about 90 degrees. In other implementations, otherorientations of the gratings may be used to create an offset of about 90degrees. For example, the gratings may be oriented such that theyproject images that are offset ±45 degrees from the x or y plane ofsample 271. The configuration of example SIM imaging system 200 may beparticularly advantageous in the case of a regularly patterned sample271 with features on a rectangular grid, as structured resolutionenhancement can be achieved using only two perpendicular gratings (e.g.,vertical grating and horizontal grating).

Gratings 230A-230B, in the example of system 200, are configured todiffract the input beams into a number of orders (e.g., 0 order, ±1orders, ±2 orders, etc.) of which the ±1 orders may be projected on thesample 271. As shown in this example, vertical grating 230A diffracts acollimated light beam into first order diffracted beams (±1 orders),spreading the first orders on the plane of the page, and horizontalgrating 230B diffracts a collimated light beam into first orderdiffracted beams, spreading the orders above and below the plane of thepage (i.e., in a plane perpendicular to the page). To improve efficiencyof the system, the zeroth order beams and all other higher order beams(i.e., ±2 orders or higher) may be blocked (i.e., filtered out of theillumination pattern projected on the sample 271). For example, a beamblocking element (not shown) such as an order filter may be insertedinto the optical path after each diffraction grating to block the0-order beam and the higher order beams. In some implementations,diffraction gratings 230A-230B may configured to diffract the beams intoonly the first orders and the 0-order (undiffracted beam) may be blockedby some beam blocking element.

Each arm includes an optical phase modulator or phase shifter 240A-240Bto phase shift the diffracted light output by each of gratings 230. Forexample, during structured imaging, the optical phase of each diffractedbeam may be shifted by some fraction (e.g., ½, ⅓, ¼, etc.) of the pitch(λ) of each fringe of the structured pattern. In the example of FIG. 2,phase modulators 240A and 240B are implemented as rotating windows thatmay use a galvanometer or other rotational actuator to rotate andmodulate the optical path-length of each diffracted beam. For example,window 240A may rotate about the vertical axis to shift the imageprojected by vertical grating 230A on sample 271 left or right, andwindow 240B may rotate about the horizontal axis to shift the imageprojected by horizontal grating 230B on sample 271 up or down.

In other implementations, further described below, other phasemodulators that change the optical path length of the diffracted light(e.g. linear translation stages, wedges, etc.) may be used.Additionally, although optical phase modulators 240A-240B areillustrated as being placed after gratings 230A-230B, in otherimplementations they may be placed at other locations in theillumination system. In some implementations, a single phase modulatormay be operated in two different directions for the different fringepatterns, or a single phase modulator may use a single motion to adjustboth of the path lengths, as described below.

In example system 200, a mirror 260 with holes 261 combines the two armsinto the optical path in a lossless manner (e.g., without significantloss of optical power, other than a small absorption in the reflectivecoating). Mirror 260 can be located such that the diffracted orders fromeach of the gratings are spatially resolved, and the unwanted orders canbe blocked. Mirror 260 passes the first orders of light output by thefirst arm through holes 261. Mirror 260 reflects the first orders oflight output by the second arm. As such, the structured illuminationpattern may be switched from a vertical orientation (e.g., grating 230A)to a horizontal orientation (e.g., grating 230B) by turning each emitteron or off or by opening and closing an optical shutter that directs alight source's light through the fiber optic cable. In otherimplementations, the structured illumination pattern may be switched byusing an optical switch to change the arm that illuminates the sample.

Also illustrated in example imaging system 200 are a projection lens265, a semi-reflective mirror 280, objective 270, and camera 290. Theprojection lens 265 may be utilized in conjunction with lens 250A toproject the Fourier transform of grating 230A into the entrance pupil ofthe objective lens 270. Similarly, the projection lens 265 may beutilized in conjunction with lens 250B to project the Fourier transformof grating 230B into the entrance pupil of the objective lens 270. Theprojection lens 265 may also be implemented to articulate along thez-axis to adjust the grating focus on the sample plane. Semi-reflectivemirror 280 may be a dichroic mirror to reflect structured illuminationlight received from each arm down into objective 270 for projection ontosample 271, and to pass through light emitted by sample 271 (e.g.,fluorescent light, which is emitted at different wavelengths than theexcitation) onto camera 290.

It is worth noting that the example of system 200 may provide a highoptical efficiency due to the absence of a polarizer. Additionally, theuse of unpolarized light may not have a significant impact on patterncontrast depending on the numerical aperture setting of the objective270.

It should be noted that, for the sake of simplicity, optical componentsof SIM imaging system 200 may have been omitted from the foregoingdiscussion. Additionally, although system 200 is illustrated in thisexample as a single channel system, in other implementations, it may beimplemented as a multi-channel system (e.g., by using two differentcameras and light sources that emit in two different wavelengths).

FIG. 3 is an optical diagram illustrating another example opticalconfiguration of a two-arm SIM imaging system 300 in accordance withsome implementations described herein. In system 300, a large, rotatingoptical window 310 may be placed after mirror 260 with holes 261. Inthis case, window 310 may be used in place of windows 240A and 240B tomodulate the phases of both sets of diffracted beams output by thevertical and horizontal diffraction gratings. Instead of being parallelwith respect to the optical axis of one of the gratings, the axis ofrotation for the rotating window 310 may be offset 45 degrees (or someother angular offset) from the optical axis of each of the vertical andhorizontal gratings to allow for phase shifting along both directionsalong one common axis of rotation of window 310. In someimplementations, the rotating window 310 may be replaced by a wedgedoptic rotating about the nominal beam axis.

FIG. 4 is an optical diagram illustrating another example opticalconfiguration of a two-arm SIM imaging system 400 in accordance withsome implementations described herein. In system 400, gratings 230A and230B are mounted on respective linear motion stages 410A and 410B thatmay be translated to change the optical path length (and thus the phase)of light emitted by gratings 230A and 230B. The axis of motion of linearmotion stages 410A-410B may be perpendicular or otherwise offset fromthe orientation of their respective grating to realize translation ofthe grating's pattern along a sample 271. In implementations, stages410A and 410B may each utilize crossed roller bearings, a linear motor,a high-accuracy linear encoder, and/or other technologies to provideprecise linear translations of the gratings to phase shift the projectedimages.

FIG. 5 is an operational flow diagram illustrating an example method 500that may be performed by a multi-arm SIM imaging system during oneimaging cycle to use structured light to create a high-resolution imagein accordance with some implementations described herein. Inimplementations, method 500 may be performed to image an entire sampleor a location of a larger sample. Method 500 will be described inconjunction with FIG. 6, which illustrates simplified illuminationfringe patterns that may be projected onto the plane of a sample 271 bya vertical grating and horizontal grating of a two-arm SIM imagingsystem during image capture. For example, SIM imaging system 200 may usevertical grating 230A and horizontal grating 230B to generate thehorizontal and vertical illumination patterns shown in FIG. 6, whilephase modulators 230A and 230B may be set to three different positionsto produce the three phase shifts shown.

At operation 510, a first arm corresponding to a first gratingorientation is turned on to begin generating illumination patterns usingthe first arm. For instance, in the implementation of imaging system200, a high-speed shutter positioned in the path between optical fiber210A and a light source may be opened or otherwise actuated such thatthe light source is not blocked. Alternatively, one or more lightsources may be turned on or off (e.g., pulsed), or an optical switch maybe used to direct a light source through the optical path of the firstarm (e.g., through one of the first or second emitter). In someinstances, operation 510 may also include turning on the light source(e.g., in the case of the first imaging cycle).

Once the first arm is turned on, at operation 520 a first gratingpattern may be projected on the sample and an image may be captured. Forexample, as illustrated by FIG. 6, vertical grating 230A may projectfirst-order illumination fringes on sample 271. Any light emitted by thesample may be captured by camera 290 and a first phase image of thefirst pattern (e.g., vertical pattern) may be captured. For instance,fluorescent dyes situated at different features of the sample 271 mayfluoresce and the resultant light may be collected by the objective lens270 and directed to an image sensor of camera 290 to detect theflorescence.

If additional phase shifted images need to be captured (decision 530),at operation 540 the pattern projected by the grating may be phaseshifted to capture the next phase image of the pattern. For example, inthe implementation of system 200, the phase of the pattern projected byvertical grating 230A may be phase shifted by rotating optical window240A. Alternatively, other optical phase modulators such as translationstages or rotating optical wedges may be used to shift the phase. Forinstance, as illustrated in the example of FIG. 6, the phase may beshifted by ⅓ of the pitch (λ) of the fringe pattern such that thepattern projected on the sample is offset by ⅓λ from the prior imagethat was captured. In some implementations, the pattern projected by thegrating may be phase shifted by moving the sample (e.g., using a motionstage) while the projected fringes remain stationary. In someimplementations, the pattern projected by the grating may be phaseshifted by moving both the sample and the projected fringes. Operations520-540 may iterate until all phase images of a first pattern arecaptured (e.g., three phase-shifted images of the vertical pattern inthe case of FIG. 6).

Once all phase images of a pattern have been captured, at operation 560the second arm corresponding to a second grating orientation of the SIMimaging system may be turned on. For instance, in the implementation ofimaging system 200, a high-speed shutter positioned in the path betweenoptical fiber 210B and a light source may be opened or otherwiseactuated such that the light source is not blocked. Alternatively, oneor more light sources may be turned on or off (e.g., pulsed), or anoptical switch may be used to direct a light source through the opticalpath of the second arm. Additionally, the other arm may be turned off. Aseries of phase images may then be captured for the next arm byrepeating operations 520-540. For instance, as illustrated by FIG. 6,horizontal grating 230B may project first-order illumination fringes onsample 271, and the projected fringes may be shifted in position by ⅓λto capture three phase images of the horizontal pattern. As anotherexample, the pattern projected by the grating may be phase shifted bymoving the sample (e.g., using a motion stage) while the projectedfringes remain stationary, or by moving both the sample and theprojected fringes.

Once all images have been captured for the imaging cycle, at operation570, a high resolution image may be constructed from the capturedimages. For example, a high resolution image may be reconstructed fromthe six images shown in FIG. 6. Suitable algorithms may be used tocombine these various images to synthesize a single image of the samplewith significantly better spatial resolution than any of the individualcomponent images.

It should be noted that although method 500 has been primarily describedin the context of single channel imaging (e.g., imaging a sample using alight source having a single wavelength), in some implementations method500 may be adapted for multi-channel imaging (e.g., imaging a sampleusing light sources having different wavelengths). In suchimplementations, method 500 may be repeated for each channel of theimaging system (e.g., sequentially, or in parallel) to generate highresolution images for each channel.

Although implementations of the two-arm SIM imaging system 200 describedherein have so far been described in the context of system 200 thatutilizes a mirror 260 with holes 261 to losslessly combine the opticalpaths of the two arms, in an alternative implementation, the two imagesof the horizontal and vertical gratings 230A-230B may be losslesslycombined by using a polarizing beam splitter in place of the mirror withholes and to illuminate the vertical grating with vertically-polarizedlight and the horizontal grating with horizontally-polarized light. Insuch implementations, the structured illumination pattern can beswitched from horizontal to vertical by turning the correspondingpolarized illumination sources on and off.

By way of example, FIG. 7 illustrates an example experimental design ofa two-arm SIM imaging system 700 that uses a polarizing beam splitter tocombine the optical paths of the arms, and that illuminates a verticalgrating with vertically-polarized light and a horizontal grating withhorizontally-polarized light. In the implementation of FIG. 7, thehorizontal and vertical gratings are G1 and G2, the horizontal andvertical rotating windows are W1 and W2, and the polarizing beamsplitter for combining the horizontal and vertical grating images isPBS2. The output of a fiber-coupled mode-scrambled multi-mode laser isFiber1.

FIG. 8A illustrates an afocal mirror image and fluorescent slidecaptured using example SIM imaging system 700, using a 20×/0.75 NAmicroscope. The afocal mirror image has fringe visibility of 84%. Thefluorescent slide image has fringe visibility of 6.6%.

FIG. 8B illustrates fringe modulation measurements acquired using system700 with a beaded flowcell. The graph illustrates typical feature imageintensity changes during a phase adjustment cycle, as the angle ofparallel plate W2 of FIG. 7 is changed.

FIG. 9 illustrates another example optical configuration of a two-armSIM imaging system 900 in accordance with some implementations describedherein. The first arm of system 900 includes a light emitter 910A (e.g.,optical fiber), an optical collimator 920A to collimate light output bylight emitter 910A, a diffraction grating 930A in a first orientationwith respect to the optical axis, and a relay lens 940A. The second armof system 900 includes a light emitter 910B, an optical collimator 920Bto collimate light output by light emitter 910B, a diffraction grating930B in a second orientation with respect to the optical axis, and arelay lens 940B.

System 900 also includes a beam combining element 950 for combining thetwo arms of the optical system. As illustrated, beam combining element950 includes a 45° prism with holes to pass through structured lightfrom the second arm of the system and a mirrored surface for reflectingstructured light received from the first arm. Before entering beamcombining element 950, each structured beam of light passes through aspatial filter having a pair of apertures to pass the ±1 orders andblock other orders. Structured light emanating from the first arm in afirst plane may be directed by reflective optic 945 into beam combingelement 950. In system 900, parallel plate optical element 960 serves asa phase adjuster and may be rotated to shift structured light in eitherorientation after beam combining element 950.

Although implementations described herein have so far been described inthe context of a two-arm structured illumination imaging system thatincludes two gratings oriented at two different angles, it should benoted that in other implementations, systems with more than two arms maybe implemented. In the case of a regularly patterned sample withfeatures on a rectangular grid, resolution enhancement can be achievedwith only two perpendicular angles (e.g., vertical grating andhorizontal grating) as described above. On the other hand, for imageresolution enhancement in all directions for other samples (e.g.,hexagonally patterned samples), three grating angles may be used. Forexample, a three-arm system may include three light emitters and threefixed diffraction gratings (one per arm), where each diffraction gratingis oriented around the optical axis of the system to project arespective pattern orientation on the sample (e.g., a 0° pattern, a 120°pattern, or a 240° pattern). In such systems, additional mirrors withholes may be used to combine the additional images of the additionalgratings into the system in a lossless manner. Alternatively, suchsystems may utilize one or more polarizing beam splitters to combine theimages of each of the gratings.

Multiple Optical Grating Slide Structured Illumination MicroscopyImaging System

In accordance with some implementations of the technology disclosedherein, the SIM imaging system may be implemented as a multiple opticalgrating slide SIM imaging system, where one linear motion stage ismounted with a plurality of diffraction gratings (or other beamsplitting optical elements) having a corresponding, fixed orientationwith respect to the optical axis of the system.

FIGS. 10A-10B are schematic diagrams illustrating an example opticalconfiguration of a dual optical grating slide SIM imaging system 1000 inaccordance with some implementations described herein. As furtherdescribed below, in the configuration of system 1000 all changes to thegrating pattern projected on sample 1070 (e.g., pattern phase shifts orrotations) may be made by linearly translating a motion stage 1030 alonga single axis of motion, to select a grating 1031 or 1032 (i.e., selectgrating orientation) or to phase shift one of gratings 1031-1032.

System 1000 includes a light emitter 1010 (e.g., optical fiber opticallycoupled to a light source), a first optical collimator 1020 (e.g.,collimation lens) to collimate light output by light emitter 1010, alinear motion stage 1030 mounted with a first diffraction grating 1031(e.g., horizontal grating) and a second diffraction grating 1032 (e.g.vertical grating), a projection lens 1040, a semi-reflective mirror 1050(e.g., dichroic mirror), an objective 1060, a sample 1070, and a camera1080. For simplicity, optical components of SIM imaging system 1000 maybe omitted from FIG. 10A. Additionally, although system 1000 isillustrated in this example as a single channel system, in otherimplementations, it may be implemented as a multi-channel system (e.g.,by using two different cameras and light sources that emit in twodifferent wavelengths).

As illustrated by FIG. 10A, a grating 1031 (e.g., a horizontaldiffraction grating) may diffract a collimated light beam into firstorder diffracted light beams (on the plane of the page). As illustratedby FIG. 10B, a diffraction grating 1032 (e.g., a vertical diffractiongrating) may diffract a beam into first orders (above and below theplane of the page). In this configuration only a single optical armhaving a single emitter 1010 (e.g., optical fiber) and single linearmotion stage is needed to image a sample 1070, which may provide systemadvantages such as reducing the number of moving system parts to improvespeed, complexity and cost. Additionally, in system 1000, the absence ofa polarizer may provide the previously mentioned advantage of highoptical efficiency. The configuration of example SIM imaging system 200may be particularly advantageous in the case of a regularly patternedsample 1070 with features on a rectangular grid, as structuredresolution enhancement can be achieved using only two perpendiculargratings (e.g., vertical grating and horizontal grating).

To improve efficiency of the system, the zeroth order beams and allother higher order diffraction beams (i.e., ±2 orders or higher) outputby each grating may be blocked (i.e., filtered out of the illuminationpattern projected on the sample 1070). For example, a beam blockingelement (not shown) such as an order filter may be inserted into theoptical path after motion stage 1030. In some implementations,diffraction gratings 1031-1032 may configured to diffract the beams intoonly the first orders and the 0-order (undiffracted beam) may be blockedby some beam blocking element.

In the example of system 1000, the two gratings may be arranged about±45° from the axis of motion (or other some other angular offset fromthe axis of motion such as about +40°/−50°, about +30°/−60°, etc.) suchthat a phase shift may be realized for each grating 1031-1032 along asingle axis of linear motion. In some implementations, the two gratingsmay be combined into one physical optical element. For example, one sideof the physical optical element may have a grating pattern in a firstorientation, and an adjacent side of the physical optical element mayhave a grating pattern in a second orientation orthogonal to the firstorientation.

Single axis linear motion stage 1030 may include one or more actuatorsto allow it to move along the X-axis relative to the sample plane, oralong the Y-axis relative to the sample plane. During operation, linearmotion stage 1030 may provide sufficient travel (e.g., about 12-15 mm)and accuracy (e.g., about less than 0.5 micrometer repeatability) tocause accurate illumination patterns to be projected for efficient imagereconstruction. In implementations where motion stage 1030 is utilizedin an automated imaging system such as a fluorescence microscope, it maybe configured to provide a high speed of operation, minimal vibrationgeneration and a long working lifetime. In implementations, linearmotion stage 1030 may include crossed roller bearings, a linear motor, ahigh-accuracy linear encoder, and/or other components. For example,motion stage 1030 may be implemented as a high-precision stepper orpiezo motion stage that may be translated using a controller.

FIG. 11 is an operational flow diagram illustrating an example method1100 that may be performed by a multiple optical grating slide SIMimaging system during one imaging cycle to use structured light tocreate a high resolution image in accordance with some implementationsdescribed herein. Depending on the implementation, method 1100 may beperformed to image an entire sample or a location of a larger sample.Method 1100 will be described in conjunction with FIG. 12, whichillustrates simplified illumination fringe patterns that may beprojected onto the plane of a sample 1070 by a first diffraction gratingand a second diffraction grating of a dual optical grating slide SIMimaging system during image capture. For example, a SIM imaging system1000 may use a first diffraction grating 1031 and second diffractiongrating 1032 to generate the illumination patterns shown in FIG. 12. Asillustrated in the example of FIG. 12, the two gratings projectperpendicular fringe patterns on the surface of sample 1070 and arearranged about ±45° from the axis of motion of linear motion stage 1030.

At operation 1110, the light source is turned on. For example, anoptical shutter may be actuated to optically couple the optical fiber oflight emitter 1010 to a light source. As another example, a light sourcemay be pulsed or an optical switch may be used to direct a light sourcethrough the optical path of the light emitter. At operation 1120, afirst grating pattern may be projected on the sample and an image may becaptured. For example, as illustrated by FIG. 12, a first grating (e.g.,grating 1031), may project first-order illumination fringes on sample1070. Any light emitted by the sample may be captured by camera 1080 anda first phase image of the first pattern (e.g., +45° pattern) may becaptured. For instance, fluorescent dyes situated at different featuresof the sample 1070 may fluoresce and the resultant light may becollected by the objective lens 1060 and directed to an image sensor ofcamera 1080 to detect the florescence.

To capture additional phase shifted images, at operation 1140 thepattern projected by the grating may be phase shifted by translating thelinear motion stage. In the example of FIG. 12, these phase shiftmotions are illustrated as steps 1 and 2. The phase shift motions mayprovide small (e.g., about 3 to 5 micrometers or smaller) moves of thegratings to slightly shift the fringe pattern projected on the grating.

By way of particular example, consider the case where the pitch λ of thefringe at the sample of FIG. 11 is 2100 nm. In this case, three phaseshifted images are captured in the sample, requiring phase shifts of theprojected fringes of λ/3, or 700 nm. Assuming an objective illuminationmagnification of 10×, the phase shift steps (linear translations)required of the single axis linear motion stage may be calculated as 700nm*10*sqrt(2), or about 9.9 μm. In this case, the sqrt(2) factoraccounts for the 45 degree offset between the orientation of the gratingand the axis of motion of the motion stage, More generally, thetranslation distance of the linear motion stage during each phase shiftstep in this example configuration may be described by λ/3×MAG×√{squareroot over (2)} where MAG is the illumination magnification.

Following capture of all phase shifted images for a diffraction grating(decision 1130), at operation 1160 the system may switch diffractiongratings by translating the linear motion stage to optically coupleanother diffraction grating to the light source of the imaging system(e.g., transition from FIG. 10A to FIG. 10B). This motion is illustratedas step 3 in the example of FIG. 12. In the case of diffraction gratingchanges, the linear motion stage may provide a relatively largetranslation (e.g., on the order of 12-15 mm).

A series of phase images may then be captured for the next grating byrepeating operations 1120-1140. For instance, as illustrated by FIG. 12,a second diffraction grating may project first-order illuminationfringes on sample 271, and the projected fringes may be shifted inposition by λ/3 by translating the linear motion stage to capture threephase images of the grating's pattern (e.g., steps 4 and 5 of FIG. 12).

Once all images have been captured for the imaging cycle, at operation1170, a high resolution image may be constructed from the capturedimages. For example, a high resolution image may be reconstructed fromthe six images shown schematically in FIG. 12. As the foregoing exampleillustrates, a multiple optical grating slide SIM imaging systemadvantageously may switch between fringe angles and phases with a singlelinear actuator, thereby saving on cost and complexity of the structuredillumination imaging system.

FIG. 13 is a diagram illustrating an example dual optical grating slideSIM imaging configuration 1300. As illustrated, the configuration 1300may include an optical fiber 1310 to emit light, a collimator 1320, alinear motion stage 1330 mounted with first and second diffractiongratings 1331-1332, a projection lens 1340, and a turning mirror 1350.In this example, gratings 1331-1332 are embedded in the same object,adjacent to each other along the axis of motion of stage 1330. Othercomponents not shown may be similar to those in FIG. 10A, such asdichroic mirror 1050, objective 1060 and sample 1070.

In some implementations, the linear motion stage or slide of the dualoptical grating slide SIM imaging system may be mounted with one or moreadditional lower frequency patterns to aid with alignment of the fringepattern that is projected on the sample by the imaging gratings (e.g.,the two gratings arranged at about ±45° from the axis of motion oflinear motion stage). For example, linear motion stage 1030 of FIGS.10A-10B may be mounted with the additional alignment pattern, or linearmotion stage 1330 of FIG. 13 may be mounted with the additionalalignment pattern. In instances where the two imaging gratings areembedded in the same substrate as depicted in FIG. 13, the alignmentgrating may also be embedded in that substrate, or it may beincorporated in a separate substrate. The alignment pattern may beplaced between the two imaging gratings or in some other suitableposition on the motion stage.

The alignment pattern, when illuminated, may be configured to project apattern having a lower frequency and/or greater pitch on a sample. Byvirtue of these characteristics, coarse alignment of the gratings withrespect to the sample may be facilitated. The alignment pattern may beimplemented as parallel lines, orthogonal lines, and/or a grating havinga lower frequency of slits than the other gratings. In someimplementations, multiple alignment patterns may be used. FIG. 17 showsone example of an alignment pattern that may be used in implementationsof the disclosure. As illustrated in this example, an alignment patternmark 1605 is implemented on the same substrate as a grating 1615,outside of clear aperture 1625. In this example, the alignment patternis implemented as two sets of orthogonal lines. By virtue of thisimplementation, grating tilt may be checked. In some implementations,the illustrated alignment pattern may be implemented in multiple areas(e.g., four corners of a substrate).

During use, the alignment pattern may be illuminated to project apattern. The alignment pattern may be utilized during SIM imaging systemmanufacture, after field installation, or during a field serviceengineer check. In some implementations, the alignment pattern may beutilized during operation of the dual optical grating slide SIM imagingsystem. For example, the alignment pattern may be illuminated to projectan alignment pattern before imaging of a sample begins.

In some implementations of the dual optical grating slide SIM imagingsystem, an optical phase modulator (e.g., a rotating window) that is aseparate component than the linear motion stage may be utilized forphase tuning. In such implementations, the optical phase modulator maybe used for phase tuning instead of the linear motion stage (e.g., thelinear motion stage may only be used for switching between the twogratings). By virtue of such implementations, the speed, accuracy,and/or reliability of the system may potentially be improved bysubstantially decreasing the number of translations required over timeby the motion stage and by obviating the need to use a motion stage tomake fine translations (e.g., on the order of μm) to select a phase.

The optical phase modulator may be placed in the light path between thelight source and sample, after the gratings (e.g., directly after themotion stage). FIG. 19 illustrates some components of one example dualoptical grating slide SIM imaging system 1900 in accordance with suchimplementations. As shown, system 1900 includes a light emitter 1910(e.g., optical fiber optically coupled to a light source), a firstoptical collimator 1920 (e.g., collimation lens) to collimate lightoutput by light emitter 1910, a linear motion stage 1930 mounted with afirst diffraction grating 1931 (e.g., horizontal grating) and a seconddiffraction grating 1932 (e.g. vertical grating), and an optical phasemodulator 1940 to phase shift the diffracted light output by eachgrating.

Pattern Angle Spatial Selection Structured Illumination MicroscopyImaging System

In accordance with some implementations of the technology disclosedherein, the SIM imaging system may be implemented as a pattern anglespatial selection SIM imaging system, whereby a fixed two dimensionaldiffraction grating is used in combination with a spatial filter wheelto project one-dimensional diffraction patterns on the sample.

FIG. 14 is a schematic diagram illustrating an example opticalconfiguration of a pattern angle spatial selection SIM imaging system1400 in accordance with some implementations described herein. Forsimplicity, optical components of SIM imaging system 1400 may be omittedfrom FIG. 14. Additionally, although system 1400 is illustrated in thisexample as a single channel system, in other implementations, it may beimplemented as a multi-channel system (e.g., by using two differentcameras and light sources that emit in two different wavelengths).

As illustrated, system 1400 includes a light emitter 1410 (e.g., opticalfiber), a collimator 1420 to collimate light emitted by emitter 1410, atwo-dimensional grating 1430, a zero order beam blocker 1440, an opticalphase modulator 1450, a projection lens 1460, a spatial filter wheel1470, a dichroic mirror 1480, an objective 1490, a sample 1491, and acamera 1495.

In this example configuration, grating 1430 is a two-dimensionaltransmission diffraction grating configured to diffract an input beaminto a number of orders (e.g., 0 order, ±1 orders, ±2 orders, etc.) intwo perpendicular directions. To improve the efficiency and performanceof the system, the zeroth order beams and all other higher order beams(i.e., ±2 orders or higher) may be blocked (i.e., filtered out of theillumination pattern projected on the sample 1491). While higher ordersmay be diffracted out at wide angles where they may be filtered using avariety of filtering elements, the 0-order component pass throughstraight through the grating in the beam path toward the sample. Toblock the 0-order component, a beam blocking element 1440 may beinserted into the optical path after two-dimensional diffraction grating1430. For example, beam blocking element 1440 may be a Volume BraggGrating (VBG), a diffractive optical element that can be patterned toreflect light normal to the element (e.g., 0-order light) and passthrough light at other angles, such as the +1 & −1 orders. With the0-order removed, smaller and more compact optics can be used to focusthe +1 & −1 orders down to the objective lens.

Optical phase modulator 1450 (e.g., a rotating window) may be used tochange the phase of the incident light to adjust the pattern phaseposition on the sample 1491. For example, optical phase modulator 1450may include a variety of moving optical elements, including a parallelplate optic tilted at a variable angle to the optical axis, a wedgedoptic rotated about the optical axis, a mirror tilted to translate thebeam, electro-optical elements, or acousto-optical elements. In oneparticular implementation, optical phase modulator 1450 may beimplemented as a parallel plate optic tilted in two perpendiculardirections to adjust the phase of two different grating angle patterns.Alternatively, in some implementations, the pattern phase position maybe adjusted by moving the sample (e.g., using a motion stage) while theprojected pattern remains stationary, or by moving both the sample andthe projected pattern.

In the example of system 1400, a rotatable spatial filter wheel 1470 mayinclude a plurality of holes oriented in two perpendicular directions(e.g., a vertical set of holes 1471 and a horizontal set of holes 1472)for selecting a vertical grating image or a horizontal grating image forprojection on the sample 1491. For example, by rotating the spatialfilter wheel, the +/−1 orders of one of the grating patterns may passthrough one of the set of holes to generate a horizontal or verticalfringed pattern on sample 1491. In implementations, spatial filter wheel1470 may be implemented as a lightweight mask or spatial filter (e.g., arotating disk including a plurality of ports or apertures).

In the configuration of system 1400, the primary optical components ofsystem 1400 may remain stationary, which may improve the stability ofthe optical system (and of the illumination pattern) and minimize theweight, vibration output and cost of the moving elements. As some of thebeam intensity (e.g., up to 50%) may need to be filtered out in eitherorientation of spatial filter wheel 1470, in some implementations thespatial filter may be configured to reflect the unneeded beams (e.g.,orders of diffraction grating pattern that is not passed through) into abeam dump for proper heat management.

FIG. 15 is a schematic diagram illustrating another example opticalconfiguration of a pattern angle spatial selection SIM imaging system1500 in accordance with some implementations described herein. Inexample imaging system 1500, the functions of the two-dimensionaltransmission grating and beam blocking element may be integrated into asolid optic 1510. Additionally, the function of a projection lens may beintegrated into solid optic 1510. In this example implementation, atwo-dimensional transmission grating 1511 is fabricated on or otherwisedisposed over a face of optic 1510 that receives collimated light fromemitter 1410 (the input of optic 1510). The dispersion angles of thegrating 1511 may be arranged such that the 0-order light can be blockedon the far side of the optic. The desired +1 & −1 orders, in bothdirections, may exit from optic 1510 through angled faces 1512 (theoutput of optic 1510) that diffract the +1 & −1 orders in an opticallydesirable direction. These output faces may include diffractive focusinglenses. Alternatively, a separate optic may be used as a projection lensto focus the beams onto the objective 1490. In system 1500, a phaseshifter 1450 and rotating spatial filter mask 1470 may be used asdescribed above.

FIG. 16 is a schematic diagram illustrating another example opticalconfiguration of a pattern angle spatial selection SIM imaging system1600 in accordance with some implementations described herein. Inexample imaging system 1600, a solid optic 1610 again may be used tointegrate the functions of a two-dimensional grating and a beam blockingelement. Additionally it may integrate the function of a projectionlens. In contrast to example imaging system 1600, the input of solidoptic 1610 is an inlet window or aperture 1614 that guides receivedlight to a two-dimensional reflective grating 1611. As grating 1611 isreflective in this example, the 0-order light may be reflected back outthrough inlet window 1614. The desired +1 & −1 orders of diffractedlight, in each of the perpendicular directions, may reflect off ofrespective reflectively-coated internal faces 1613 of the optic 1610,and exit through outlet faces 1612. In implementations, these outletfaces may include diffractive focusing lenses. Alternatively, a separateprojection lens optic 1615 may be used to focus the beams onto theobjective 1490. In system 1600, a phase shifter 1450 and rotatingspatial filter mask 1470 may be used as described above.

Although some implementations of the disclosure have been illustrated inthe figures in the context of SIM imaging systems that use one or moreoptics to reimage collected excitation light (e.g., light recollected bythe objective) onto an image sensor (e.g., a CCD camera sensor), itshould be appreciated that the various implementations described hereinmay apply to SIM imaging systems that utilize an image sensor (e.g., aCMOS sensor) that is in an active plane of an imaged sample. By way ofillustrative example, FIG. 18, illustrates a sample 1710 that may beformed over an image sensor assembly 1740 of a SIM imaging system, inaccordance with some implementations described herein. For example,features of the sample may be photolithographically aligned with pixelsof the image sensor. Any light emitted by patterned sample 1710 inresponse to structured illumination is collected by image sensorassembly 1740, which is positioned directly below sample 1710 in thisexample. Forming sample 1710 over image sensor assembly 1740 may providethe advantage of ensuring that patterned features 1711 of the sample1710 remain aligned relative to particular photosites (e.g., pixels) ofimage sensor assembly 1740 during imaging.

Sample 1710 may be patterned and aligned with image sensor assembly 1740such that each light sensor (e.g., pixel) of image sensor 1740 has oneor more features 1711 formed and/or mounted above it. As illustrated inthe example of FIG. 18, sample 1710 is patterned over image sensorassembly 1740 such that one feature 1711 is formed over each pixel ofthe pixel array of image sensor assembly 1740. In other implementations,more than one feature may be formed over each pixel.

In the case of a fluorescent sample, for instance, illuminated features1711 of the sample may fluoresce in response to the structuredexcitation light 1760, and the resultant light 1761 emitted by features1711 may be collected by photosites (e.g., pixels) of image sensorassembly 1740 to detect fluorescence. For example, as illustrated byFIG. 18, pixels (1,1) and (1,3) of image sensor assembly 1740 maycollect light 1761 that is emitted by the feature 1711 of the samplethat is positioned or patterned over it. In some implementations, alayer (not shown) may provide isolation between sample 1710 and imagesensor assembly 1740 (e.g., to shield the image sensor assembly from afluidic environment of the sample). In other implementations, sample1710 may be mounted and aligned over image sensor assembly 1740.

It should be noted that although FIG. 18 illustrates an examplerepresentation of a SIM imaging system where the SIM fringes line upwith the features of the sample in the correct orientation, in practicethis is not necessarily or typically the case for SIM imaging. Forexample, over time and/or space, there may be drift in the spacingbetween adjacent fringes, the phase or angle of the structuredillumination pattern, and/or the orientation of the fringe patternrelative to the illuminated sample. Owing to these variations in SIMparameters, in some instances some illuminated features may be 80% “on”while other features may be 60% “on” and yet other features may be 20%“on.” As such, it should be appreciated that in such systems, SIMimaging algorithms may be utilized to take into account these processvariations during image reconstruction. For example, variations instructured illumination parameters may be estimated and/or predictedover time to account for these variations.

As used herein, the term module might describe a given unit offunctionality that can be performed in accordance with one or moreimplementations of the present application. As used herein, a modulemight be implemented utilizing any form of hardware, software, or acombination thereof. For example, one or more processors, controllers,ASICs, PLAs, PALs, CPLDs, FPGAs, logical components, software routinesor other mechanisms might be implemented to make up a module. Inimplementation, the various modules described herein might beimplemented as discrete modules or the functions and features describedcan be shared in part or in total among one or more modules. In otherwords, as would be apparent to one of ordinary skill in the art afterreading this description, the various features and functionalitydescribed herein may be implemented in any given application and can beimplemented in one or more separate or shared modules in variouscombinations and permutations. Even though various features or elementsof functionality may be individually described or claimed as separatemodules, one of ordinary skill in the art will understand that thesefeatures and functionality can be shared among one or more commonsoftware and hardware elements, and such description shall not requireor imply that separate hardware or software components are used toimplement such features or functionality.

In this document, the terms “computer readable medium”, “computer usablemedium” and “computer program medium” are used to generally refer tonon-transitory media, volatile or non-volatile, such as, for example, amemory, storage unit, and media. These and other various forms ofcomputer program media or computer usable media may be involved incarrying one or more sequences of one or more instructions to aprocessing device for execution. Such instructions embodied on themedium, are generally referred to as “computer program code” or a“computer program product” (which may be grouped in the form of computerprograms or other groupings).

Although described above in terms of various example implementations andimplementations, it should be understood that the various features,aspects and functionality described in one or more of the individualimplementations are not limited in their applicability to the particularimplementation with which they are described, but instead can beapplied, alone or in various combinations, to one or more of the otherimplementations of the application, whether or not such implementationsare described and whether or not such features are presented as being apart of a described implementation. Thus, the breadth and scope of thepresent application should not be limited by any of the above-describedexample implementations.

It should be appreciated that all combinations of the foregoing concepts(provided such concepts are not mutually inconsistent) are contemplatedas being part of the inventive subject matter disclosed herein. Inparticular, all combinations of claimed subject matter appearing at theend of this disclosure are contemplated as being part of the inventivesubject matter disclosed herein.

The terms “substantially” and “about” used throughout this disclosure,including the claims, are used to describe and account for smallfluctuations, such as due to variations in processing. For example, theycan refer to less than or equal to ±5%, such as less than or equal to±2%, such as less than or equal to ±1%, such as less than or equal to±0.5%, such as less than or equal to ±0.2%, such as less than or equalto ±0.1%, such as less than or equal to ±0.05%.

To the extent applicable, the terms “first,” “second,” “third,” etc.herein are merely employed to show the respective objects described bythese terms as separate entities and are not meant to connote a sense ofchronological order, unless stated explicitly otherwise herein.

Terms and phrases used in this document, and variations thereof, unlessotherwise expressly stated, should be construed as open ended as opposedto limiting. As examples of the foregoing: the term “including” shouldbe read as meaning “including, without limitation” or the like; the term“example” is used to provide some instances of the item in discussion,not an exhaustive or limiting list thereof; the terms “a” or “an” shouldbe read as meaning “at least one,” “one or more” or the like; andadjectives such as “conventional,” “traditional,” “normal,” “standard,”“known” and terms of similar meaning should not be construed as limitingthe item described to a given time period or to an item available as ofa given time, but instead should be read to encompass conventional,traditional, normal, or standard technologies that may be available orknown now or at any time in the future. Likewise, where this documentrefers to technologies that would be apparent or known to one ofordinary skill in the art, such technologies encompass those apparent orknown to the skilled artisan now or at any time in the future.

The presence of broadening words and phrases such as “one or more,” “atleast,” “but not limited to” or other like phrases in some instancesshall not be read to mean that the narrower case is intended or requiredin instances where such broadening phrases may be absent. The use of theterm “module” does not imply that the components or functionalitydescribed or claimed as part of the module are all configured in acommon package. Indeed, any or all of the various components of amodule, whether control logic or other components, can be combined in asingle package or separately maintained and can further be distributedin multiple groupings or packages or across multiple locations.

Additionally, the various implementations set forth herein are describedin terms of example block diagrams, flow charts and other illustrations.As will become apparent to one of ordinary skill in the art afterreading this document, the illustrated implementations and their variousalternatives can be implemented without confinement to the illustratedexamples. For example, block diagrams and their accompanying descriptionshould not be construed as mandating a particular architecture orconfiguration.

While various implementations of the present disclosure have beendescribed above, it should be understood that they have been presentedby way of example only, and not of limitation. Likewise, the variousdiagrams may depict an example architectural or other configuration forthe disclosure, which is done to aid in understanding the features andfunctionality that can be included in the disclosure. The disclosure isnot restricted to the illustrated example architectures orconfigurations, but the desired features can be implemented using avariety of alternative architectures and configurations. Indeed, it willbe apparent to one of skill in the art how alternative functional,logical or physical partitioning and configurations can be implementedto implement the desired features of the present disclosure. Also, amultitude of different constituent module names other than thosedepicted herein can be applied to the various partitions. Additionally,with regard to flow diagrams, operational descriptions and methodclaims, the order in which the steps are presented herein shall notmandate that various implementations be implemented to perform therecited functionality in the same order unless the context dictatesotherwise.

What is claimed is:
 1. A structured illumination imaging system,comprising: a light emitter to emit light; a two-dimensional diffractiongrating to diffract light emitted by the light emitter to project afirst plurality of fringes oriented in a first direction on a sampleplane and to project a second plurality of fringes oriented in a seconddirection, perpendicular to the first direction, on the sample plane;and a spatial filter wheel to pass through diffracted light receivedfrom the two-dimensional diffraction grating in a respective one of thefirst or second directions and block light in a respective one of thefirst or second directions, the spatial filter wheel comprising a firstplurality of apertures to pass through light diffracted by thetwo-dimensional diffraction grating in the first direction and a secondplurality of apertures orthogonal to the first plurality of apertures topass through light diffracted by the two-dimensional diffraction gratingin the second direction.
 2. The structured illumination imaging systemof claim 1, further comprising: a beam blocking element to block 0thorder light transmitted by the two-dimensional diffraction grating. 3.The structure illumination imaging system of claim 2, wherein the beamblocking element comprises a diffractive optical element patterned toreflect light normal to the element and pass through light at otherangles.
 4. The structured illumination imaging system of claim 1,wherein the spatial filter wheel is to reflect diffraction orders oflight received from the two-dimensional diffraction grating that are notpassed through.
 5. The structured illumination imaging system of claim1, wherein the two-dimensional diffraction grating is a transmissivediffraction grating.
 6. The structured illumination imaging system ofclaim 5, further comprising: a solid optic, wherein the transmissivediffraction grating is disposed over or formed on a face of the solidoptic that receives light from the light emitter.
 7. The structuredillumination imaging system of claim 6, wherein dispersion angles of thetransmissive diffraction grating are arranged such that 0th order lightis blocked on a far side of the solid optic.
 8. The structuredillumination imaging system of claim 7, wherein the solid opticcomprises angled faces to diffract and output first orders of lightdiffracted by the two-dimensional transmissive diffraction grating. 9.The structured illumination imaging system of claim 8, wherein theangled faces comprise a focusing lens.
 10. The structured illuminationimaging system of claim 8, further comprising: a projection lens toreceive light output by the solid optic.
 11. The structured illuminationimaging system of claim 1, wherein the two-dimensional diffractiongrating is a two-dimensional reflective diffraction grating.
 12. Thestructured illumination imaging system of claim 11, further comprising:a solid optic, wherein the two-dimensional reflective diffractiongrating is formed on or disposed over a face of the solid optic oppositean aperture of the solid optic that receives light from the lightemitter.
 13. The structured illumination imaging system of claim 12,wherein the solid optic further comprises reflective internal faces toreflect and output first orders of light diffracted by thetwo-dimensional reflective diffraction grating through outlet faces ofthe solid optic.
 14. The structured illumination imaging system of claim13, wherein the outlet faces comprise a diffractive focusing lens. 15.The structure illumination imaging system of claim 13, furthercomprising: a projection lens to receive light output by the solidoptic.
 16. The structured illumination imaging system of claim 1,further comprising: one or more optical elements to phase shift thefirst plurality of fringes and the second plurality of fringes, whereinthe one or more optical elements to phase shift the first plurality offringes and the second plurality of fringes comprise a parallel plateoptic tilted in two perpendicular directions.
 17. A method for using astructured illumination imaging system, comprising: turning on a lightemitter of a structured illumination imaging system, the structuredillumination imaging system comprising a two-dimensional diffractiongrating; receiving light emitted by the light emitter at thetwo-dimensional diffraction grating to output first diffracted lightoriented in a first direction and second diffracted light oriented in asecond direction perpendicular to the first direction; passing the firstdiffracted light through a first plurality of apertures of a spatialfilter wheel and blocking the second diffracted light at the spatialfilter wheel; projecting the first diffracted light passed through thefirst plurality of holes as a first plurality of fringes on a sampleplane; and capturing a first plurality of phase images of light emittedby the sample, wherein during capture of the first plurality of images,the first plurality of fringes are phase shifted on the sample plane.18. The method of claim 17, further comprising: rotating the spatialfilter wheel such that it passes the second diffracted light through asecond plurality of apertures of the spatial filter wheel and blocks thefirst diffracted light at the spatial filter wheel; projecting thesecond diffracted light that passes through the second plurality ofholes as a second plurality of fringes, orthogonal to the firstplurality of fringes, on the sample plane; and capturing a secondplurality of phase images of light emitted by the sample, wherein duringcapture of the second plurality of images, the second plurality offringes are phase shifted on the sample plane.
 19. The method of claim17, wherein the two-dimensional diffraction grating is a two-dimensionaltransmissive diffraction grating formed on a face of a solid optic,wherein the method further comprises: blocking 0th order light output bythe transmissive diffraction grating at a side of the solid opticopposite the transmissive diffraction grating; and diffracting andoutputting, from angled faces of the solid optic, first orders of lightdiffracted by the two-dimensional transmissive diffraction grating. 20.The method of claim 17, wherein the two-dimensional diffraction gratingis a two-dimensional reflective diffraction grating formed on a face ofa solid optic opposite an aperture of the solid optic that receiveslight from the light emitter, wherein the method further comprises:reflecting, at faces of the solid optic, first orders of lightdiffracted by the two-dimensional reflective diffraction grating.